<!DOCTYPE html>
<html lang="zh-CN">
  <head>
    <meta charset="utf-8">
    <meta name="viewport" content="width=device-width,initial-scale=1">
    <title>HDFS数据用MapReduce导入Hbase | 智能技术</title>
    <meta name="generator" content="VuePress 1.8.2">
    <script data-ad-client="ca-pub-1977396037355374" async="true" src="https://pagead2.googlesyndication.com/pagead/js/adsbygoogle.js"></script>
    <meta name="description" content="使用MapReduce框架将HDFS上的数据导入Hbase示例代码">
    <meta name="baidu-site-verification" content="fP9EQkbGCM">
    
    <link rel="preload" href="/assets/css/0.styles.89695ccf.css" as="style"><link rel="preload" href="/assets/js/app.cee3c598.js" as="script"><link rel="preload" href="/assets/js/13.16b64097.js" as="script"><link rel="prefetch" href="/assets/js/10.81160393.js"><link rel="prefetch" href="/assets/js/11.777111f5.js"><link rel="prefetch" href="/assets/js/12.c58a9b41.js"><link rel="prefetch" href="/assets/js/14.81b84b8b.js"><link rel="prefetch" href="/assets/js/15.ae9bbcfc.js"><link rel="prefetch" href="/assets/js/16.cb71ce72.js"><link rel="prefetch" href="/assets/js/17.7eb91f50.js"><link rel="prefetch" href="/assets/js/18.1f750554.js"><link rel="prefetch" href="/assets/js/19.525a894f.js"><link rel="prefetch" href="/assets/js/2.0a188998.js"><link rel="prefetch" href="/assets/js/20.f186cf8f.js"><link rel="prefetch" href="/assets/js/21.b3e01dc7.js"><link rel="prefetch" href="/assets/js/22.ea0793c3.js"><link rel="prefetch" href="/assets/js/23.da81792d.js"><link rel="prefetch" href="/assets/js/24.0d075789.js"><link rel="prefetch" href="/assets/js/25.901ac41c.js"><link rel="prefetch" href="/assets/js/26.c3112033.js"><link rel="prefetch" href="/assets/js/27.0d727913.js"><link rel="prefetch" href="/assets/js/3.0d59f57e.js"><link rel="prefetch" href="/assets/js/4.58ac6eea.js"><link rel="prefetch" href="/assets/js/5.b48f4120.js"><link rel="prefetch" href="/assets/js/6.6a7eb352.js"><link rel="prefetch" href="/assets/js/7.0a07ef26.js"><link rel="prefetch" href="/assets/js/8.0332c4f5.js"><link rel="prefetch" href="/assets/js/9.9df962ab.js">
    <link rel="stylesheet" href="/assets/css/0.styles.89695ccf.css">
  </head>
  <body>
    <div id="app" data-server-rendered="true"><section id="global-layout" data-v-4fb7124e><header class="header" data-v-61b62cbe data-v-4fb7124e><div class="header-navbar" data-v-61b62cbe><div class="flex-xbc main header-nav" data-v-61b62cbe><div class="nav-link" data-v-61b62cbe><!----> <nav class="link-list" data-v-61b62cbe><a href="/" class="list-item router-link-active" data-v-61b62cbe>首页</a><a href="/posts/" class="list-item" data-v-61b62cbe>博客</a><a href="/category/" class="list-item" data-v-61b62cbe>分类</a><a href="/tag/" class="list-item" data-v-61b62cbe>标签</a><a href="/about/" class="list-item" data-v-61b62cbe>关于</a></nav></div> <div class="search-box" data-v-61b62cbe><input aria-label="Search" autocomplete="off" spellcheck="false" value=""> <!----></div></div></div> </header> <!----> <section class="page" data-v-4fb7124e data-v-4fb7124e><section class="info" style="background-image:url(/images/2.jpg);" data-v-52fe94f0><article class="main info-content" data-v-a5c9dc12 data-v-52fe94f0><div class="content-header" data-v-a5c9dc12><h1 class="header-title" data-v-a5c9dc12>HDFS数据用MapReduce导入Hbase</h1></div> <div class="flex-wcc content-tag" data-v-a5c9dc12><div class="inblock tag-list" data-v-a5c9dc12><a href="/category/问题解决/" class="tag-text" data-v-a5c9dc12>问题解决
      </a></div> <span class="tag-space" data-v-a5c9dc12>/</span> <div class="inblock tag-list" data-v-a5c9dc12><a href="/tag/HDFS/" class="tag-text" data-v-a5c9dc12>HDFS
      </a><a href="/tag/MapReduce/" class="tag-text" data-v-a5c9dc12>MapReduce
      </a><a href="/tag/Hbase/" class="tag-text" data-v-a5c9dc12>Hbase
      </a></div></div> <div class="content content__default" data-v-a5c9dc12><p>#对应代码如下</p> <div class="language-java extra-class"><pre class="language-java"><code><span class="token keyword">package</span> <span class="token namespace">hbase</span><span class="token punctuation">;</span>

<span class="token keyword">import</span> <span class="token namespace">org<span class="token punctuation">.</span>apache<span class="token punctuation">.</span>hadoop<span class="token punctuation">.</span>conf<span class="token punctuation">.</span></span><span class="token class-name">Configuration</span><span class="token punctuation">;</span>
<span class="token keyword">import</span> <span class="token namespace">org<span class="token punctuation">.</span>apache<span class="token punctuation">.</span>hadoop<span class="token punctuation">.</span>fs<span class="token punctuation">.</span></span><span class="token class-name">Path</span><span class="token punctuation">;</span>
<span class="token keyword">import</span> <span class="token namespace">org<span class="token punctuation">.</span>apache<span class="token punctuation">.</span>hadoop<span class="token punctuation">.</span>hbase<span class="token punctuation">.</span></span><span class="token class-name">HBaseConfiguration</span><span class="token punctuation">;</span>
<span class="token keyword">import</span> <span class="token namespace">org<span class="token punctuation">.</span>apache<span class="token punctuation">.</span>hadoop<span class="token punctuation">.</span>hbase<span class="token punctuation">.</span>client<span class="token punctuation">.</span></span><span class="token class-name">Put</span><span class="token punctuation">;</span>
<span class="token keyword">import</span> <span class="token namespace">org<span class="token punctuation">.</span>apache<span class="token punctuation">.</span>hadoop<span class="token punctuation">.</span>hbase<span class="token punctuation">.</span>io<span class="token punctuation">.</span></span><span class="token class-name">ImmutableBytesWritable</span><span class="token punctuation">;</span>
<span class="token keyword">import</span> <span class="token namespace">org<span class="token punctuation">.</span>apache<span class="token punctuation">.</span>hadoop<span class="token punctuation">.</span>hbase<span class="token punctuation">.</span>mapreduce<span class="token punctuation">.</span></span><span class="token class-name">TableMapReduceUtil</span><span class="token punctuation">;</span>
<span class="token keyword">import</span> <span class="token namespace">org<span class="token punctuation">.</span>apache<span class="token punctuation">.</span>hadoop<span class="token punctuation">.</span>hbase<span class="token punctuation">.</span>util<span class="token punctuation">.</span></span><span class="token class-name">Bytes</span><span class="token punctuation">;</span>
<span class="token keyword">import</span> <span class="token namespace">org<span class="token punctuation">.</span>apache<span class="token punctuation">.</span>hadoop<span class="token punctuation">.</span>io<span class="token punctuation">.</span></span><span class="token operator">*</span><span class="token punctuation">;</span>
<span class="token keyword">import</span> <span class="token namespace">org<span class="token punctuation">.</span>apache<span class="token punctuation">.</span>hadoop<span class="token punctuation">.</span>mapreduce<span class="token punctuation">.</span></span><span class="token class-name">Job</span><span class="token punctuation">;</span>
<span class="token keyword">import</span> <span class="token namespace">org<span class="token punctuation">.</span>apache<span class="token punctuation">.</span>hadoop<span class="token punctuation">.</span>mapreduce<span class="token punctuation">.</span></span><span class="token class-name">Mapper</span><span class="token punctuation">;</span>
<span class="token keyword">import</span> <span class="token namespace">org<span class="token punctuation">.</span>apache<span class="token punctuation">.</span>hadoop<span class="token punctuation">.</span>mapreduce<span class="token punctuation">.</span>lib<span class="token punctuation">.</span>input<span class="token punctuation">.</span></span><span class="token class-name">FileInputFormat</span><span class="token punctuation">;</span>
<span class="token keyword">import</span> <span class="token namespace">org<span class="token punctuation">.</span>apache<span class="token punctuation">.</span>hadoop<span class="token punctuation">.</span>mapreduce<span class="token punctuation">.</span>lib<span class="token punctuation">.</span>input<span class="token punctuation">.</span></span><span class="token class-name">SequenceFileInputFormat</span><span class="token punctuation">;</span>
</code></pre></div> <div class="language-java extra-class"><pre class="language-java"><code><span class="token keyword">import</span> <span class="token namespace">java<span class="token punctuation">.</span>io<span class="token punctuation">.</span></span><span class="token class-name">IOException</span><span class="token punctuation">;</span>
<span class="token keyword">import</span> <span class="token namespace">java<span class="token punctuation">.</span>text<span class="token punctuation">.</span></span><span class="token class-name">DateFormat</span><span class="token punctuation">;</span>
<span class="token keyword">import</span> <span class="token namespace">java<span class="token punctuation">.</span>text<span class="token punctuation">.</span></span><span class="token class-name">SimpleDateFormat</span><span class="token punctuation">;</span>
<span class="token keyword">import</span> <span class="token namespace">java<span class="token punctuation">.</span>util<span class="token punctuation">.</span></span><span class="token operator">*</span><span class="token punctuation">;</span>
<span class="token keyword">import</span> <span class="token namespace">java<span class="token punctuation">.</span>util<span class="token punctuation">.</span>regex<span class="token punctuation">.</span></span><span class="token class-name">Matcher</span><span class="token punctuation">;</span>
<span class="token keyword">import</span> <span class="token namespace">java<span class="token punctuation">.</span>util<span class="token punctuation">.</span>regex<span class="token punctuation">.</span></span><span class="token class-name">Pattern</span><span class="token punctuation">;</span>

<span class="token comment">/**
 * Created by lcc on 2017/10/26.
 */</span>
<span class="token keyword">public</span> <span class="token keyword">class</span> <span class="token class-name">FromWeibo</span> <span class="token punctuation">{</span>

    <span class="token keyword">public</span> <span class="token keyword">static</span> <span class="token keyword">class</span> <span class="token class-name">ToHbaseMap</span> <span class="token keyword">extends</span> <span class="token class-name">Mapper</span><span class="token generics"><span class="token punctuation">&lt;</span><span class="token class-name">BytesWritable</span><span class="token punctuation">,</span> <span class="token class-name">MapWritable</span><span class="token punctuation">,</span> <span class="token class-name">ImmutableBytesWritable</span><span class="token punctuation">,</span> <span class="token class-name">Put</span><span class="token punctuation">&gt;</span></span><span class="token punctuation">{</span>
        <span class="token keyword">protected</span> <span class="token class-name">Date</span> <span class="token function">getPostDate</span><span class="token punctuation">(</span><span class="token class-name">String</span> down<span class="token punctuation">,</span> <span class="token class-name">String</span> s<span class="token punctuation">)</span><span class="token punctuation">{</span>
            <span class="token class-name">Date</span> postdate<span class="token operator">=</span><span class="token keyword">null</span><span class="token punctuation">;</span>
            <span class="token class-name">DateFormat</span> downdf <span class="token operator">=</span> <span class="token keyword">new</span> <span class="token class-name">SimpleDateFormat</span><span class="token punctuation">(</span><span class="token string">&quot;yyyy-MM-dd HH:mm&quot;</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
            <span class="token keyword">try</span> <span class="token punctuation">{</span>
                <span class="token class-name">Date</span> downd <span class="token operator">=</span> downdf<span class="token punctuation">.</span><span class="token function">parse</span><span class="token punctuation">(</span>down<span class="token punctuation">)</span><span class="token punctuation">;</span>
                <span class="token class-name">String</span> reg <span class="token operator">=</span> <span class="token string">&quot;&lt;p class=\&quot;release_date\&quot;&gt;(.*?)&lt;/p&gt;&quot;</span><span class="token punctuation">;</span>
                <span class="token class-name">Pattern</span> pattern <span class="token operator">=</span> <span class="token class-name">Pattern</span><span class="token punctuation">.</span><span class="token function">compile</span><span class="token punctuation">(</span>reg<span class="token punctuation">)</span><span class="token punctuation">;</span>

                <span class="token class-name">Matcher</span> matcher <span class="token operator">=</span> pattern<span class="token punctuation">.</span><span class="token function">matcher</span><span class="token punctuation">(</span>s<span class="token punctuation">)</span><span class="token punctuation">;</span>
                <span class="token class-name">DateFormat</span> df <span class="token operator">=</span> <span class="token keyword">new</span> <span class="token class-name">SimpleDateFormat</span><span class="token punctuation">(</span><span class="token string">&quot;yyyy-MM-dd HH:mm&quot;</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
                <span class="token class-name">DateFormat</span> df2 <span class="token operator">=</span> <span class="token keyword">new</span> <span class="token class-name">SimpleDateFormat</span><span class="token punctuation">(</span><span class="token string">&quot;MM月dd日 HH:mm&quot;</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
                <span class="token keyword">if</span> <span class="token punctuation">(</span>matcher<span class="token punctuation">.</span><span class="token function">find</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span> <span class="token punctuation">{</span>
                    <span class="token class-name">String</span> out <span class="token operator">=</span> matcher<span class="token punctuation">.</span><span class="token function">group</span><span class="token punctuation">(</span><span class="token number">1</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
                    out <span class="token operator">=</span> out<span class="token punctuation">.</span><span class="token function">trim</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
                    <span class="token keyword">if</span> <span class="token punctuation">(</span><span class="token operator">!</span>out<span class="token punctuation">.</span><span class="token function">equals</span><span class="token punctuation">(</span><span class="token string">&quot;&quot;</span><span class="token punctuation">)</span><span class="token punctuation">)</span> <span class="token punctuation">{</span>
                        <span class="token keyword">try</span> <span class="token punctuation">{</span>
                            postdate <span class="token operator">=</span> df<span class="token punctuation">.</span><span class="token function">parse</span><span class="token punctuation">(</span>out<span class="token punctuation">)</span><span class="token punctuation">;</span>
                        <span class="token punctuation">}</span> <span class="token keyword">catch</span> <span class="token punctuation">(</span><span class="token class-name">Exception</span> e<span class="token punctuation">)</span> <span class="token punctuation">{</span>
                            <span class="token keyword">try</span> <span class="token punctuation">{</span>
                                postdate <span class="token operator">=</span> df2<span class="token punctuation">.</span><span class="token function">parse</span><span class="token punctuation">(</span>out<span class="token punctuation">)</span><span class="token punctuation">;</span>
                                postdate<span class="token punctuation">.</span><span class="token function">setYear</span><span class="token punctuation">(</span><span class="token number">117</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
                            <span class="token punctuation">}</span> <span class="token keyword">catch</span> <span class="token punctuation">(</span><span class="token class-name">Exception</span> e2<span class="token punctuation">)</span> <span class="token punctuation">{</span>
                                <span class="token class-name">String</span> reg2 <span class="token operator">=</span> <span class="token string">&quot;[1-9]\\d*&quot;</span><span class="token punctuation">;</span>
                                <span class="token class-name">Pattern</span> pattern2 <span class="token operator">=</span> <span class="token class-name">Pattern</span><span class="token punctuation">.</span><span class="token function">compile</span><span class="token punctuation">(</span>reg2<span class="token punctuation">)</span><span class="token punctuation">;</span>
                                <span class="token class-name">Matcher</span> matcher2 <span class="token operator">=</span> pattern2<span class="token punctuation">.</span><span class="token function">matcher</span><span class="token punctuation">(</span>out<span class="token punctuation">.</span><span class="token function">trim</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
                                <span class="token keyword">if</span> <span class="token punctuation">(</span>matcher2<span class="token punctuation">.</span><span class="token function">find</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span> <span class="token punctuation">{</span>
                                    <span class="token keyword">int</span> m <span class="token operator">=</span> <span class="token class-name">Integer</span><span class="token punctuation">.</span><span class="token function">parseInt</span><span class="token punctuation">(</span>matcher2<span class="token punctuation">.</span><span class="token function">group</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
                                    postdate<span class="token operator">=</span><span class="token keyword">new</span> <span class="token class-name">Date</span><span class="token punctuation">(</span>downd<span class="token punctuation">.</span><span class="token function">getTime</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token operator">-</span>m<span class="token operator">*</span><span class="token number">60</span><span class="token operator">*</span><span class="token number">1000</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
                                <span class="token punctuation">}</span> <span class="token keyword">else</span> <span class="token punctuation">{</span>
                                    postdate<span class="token operator">=</span>downd<span class="token punctuation">;</span>
                                <span class="token punctuation">}</span>
                            <span class="token punctuation">}</span>
                        <span class="token punctuation">}</span>
                    <span class="token punctuation">}</span>
                <span class="token punctuation">}</span>
            <span class="token punctuation">}</span>
            <span class="token keyword">catch</span> <span class="token punctuation">(</span><span class="token class-name">Exception</span> ex<span class="token punctuation">)</span> <span class="token punctuation">{</span>

            <span class="token punctuation">}</span>
            <span class="token keyword">return</span> postdate<span class="token punctuation">;</span>
        <span class="token punctuation">}</span>
        <span class="token keyword">protected</span> <span class="token class-name">String</span> <span class="token function">getPostContent</span><span class="token punctuation">(</span><span class="token class-name">String</span> s<span class="token punctuation">)</span> <span class="token punctuation">{</span>
            <span class="token class-name">String</span> content <span class="token operator">=</span> <span class="token string">&quot;&quot;</span><span class="token punctuation">;</span>
            <span class="token class-name">String</span> reg <span class="token operator">=</span> <span class="token string">&quot;&lt;p class=\&quot;content\&quot;&gt;(.*?)&lt;/p&gt;&quot;</span><span class="token punctuation">;</span>
            <span class="token class-name">Pattern</span> pattern <span class="token operator">=</span> <span class="token class-name">Pattern</span><span class="token punctuation">.</span><span class="token function">compile</span><span class="token punctuation">(</span>reg<span class="token punctuation">)</span><span class="token punctuation">;</span>
            <span class="token class-name">Matcher</span> matcher <span class="token operator">=</span> pattern<span class="token punctuation">.</span><span class="token function">matcher</span><span class="token punctuation">(</span>s<span class="token punctuation">)</span><span class="token punctuation">;</span>
            <span class="token keyword">if</span> <span class="token punctuation">(</span>matcher<span class="token punctuation">.</span><span class="token function">find</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span> <span class="token punctuation">{</span>
                content <span class="token operator">=</span> matcher<span class="token punctuation">.</span><span class="token function">group</span><span class="token punctuation">(</span><span class="token number">1</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
            <span class="token punctuation">}</span>
            <span class="token keyword">return</span> content<span class="token punctuation">;</span>
        <span class="token punctuation">}</span>
        <span class="token keyword">protected</span> <span class="token class-name">Date</span> <span class="token function">getRepostDate</span><span class="token punctuation">(</span><span class="token class-name">String</span> s<span class="token punctuation">)</span><span class="token punctuation">{</span>
            <span class="token class-name">Date</span> d<span class="token operator">=</span><span class="token keyword">null</span><span class="token punctuation">;</span>
            <span class="token class-name">DateFormat</span> df <span class="token operator">=</span> <span class="token keyword">new</span> <span class="token class-name">SimpleDateFormat</span><span class="token punctuation">(</span><span class="token string">&quot;yyyy-MM-dd HH:mm&quot;</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
            <span class="token keyword">try</span><span class="token punctuation">{</span>
                d<span class="token operator">=</span>df<span class="token punctuation">.</span><span class="token function">parse</span><span class="token punctuation">(</span>s<span class="token punctuation">)</span><span class="token punctuation">;</span>
            <span class="token punctuation">}</span><span class="token keyword">catch</span> <span class="token punctuation">(</span><span class="token class-name">Exception</span> e<span class="token punctuation">)</span> <span class="token punctuation">{</span>

            <span class="token punctuation">}</span>
            <span class="token keyword">return</span> d<span class="token punctuation">;</span>
        <span class="token punctuation">}</span>
        <span class="token keyword">protected</span> <span class="token keyword">void</span> <span class="token function">map</span><span class="token punctuation">(</span><span class="token class-name">BytesWritable</span> key<span class="token punctuation">,</span> <span class="token class-name">MapWritable</span> value<span class="token punctuation">,</span> <span class="token class-name">Context</span> context<span class="token punctuation">)</span> <span class="token keyword">throws</span> <span class="token class-name">IOException</span><span class="token punctuation">,</span><span class="token class-name">InterruptedException</span><span class="token punctuation">{</span>
            <span class="token class-name">Set</span><span class="token generics"><span class="token punctuation">&lt;</span><span class="token class-name">Writable</span><span class="token punctuation">&gt;</span></span> vs <span class="token operator">=</span> value<span class="token punctuation">.</span><span class="token function">keySet</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
            <span class="token class-name">Iterator</span><span class="token generics"><span class="token punctuation">&lt;</span><span class="token class-name">Writable</span><span class="token punctuation">&gt;</span></span> it <span class="token operator">=</span> vs<span class="token punctuation">.</span><span class="token function">iterator</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
            <span class="token keyword">int</span> i <span class="token operator">=</span> <span class="token number">0</span><span class="token punctuation">;</span>
            <span class="token class-name">List</span><span class="token generics"><span class="token punctuation">&lt;</span><span class="token class-name">String</span><span class="token punctuation">&gt;</span></span> outkey<span class="token operator">=</span><span class="token keyword">new</span> <span class="token class-name">LinkedList</span><span class="token generics"><span class="token punctuation">&lt;</span><span class="token class-name">String</span><span class="token punctuation">&gt;</span></span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
            <span class="token class-name">List</span><span class="token generics"><span class="token punctuation">&lt;</span><span class="token class-name">String</span><span class="token punctuation">&gt;</span></span> outvalue<span class="token operator">=</span><span class="token keyword">new</span> <span class="token class-name">LinkedList</span><span class="token generics"><span class="token punctuation">&lt;</span><span class="token class-name">String</span><span class="token punctuation">&gt;</span></span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
            <span class="token keyword">while</span> <span class="token punctuation">(</span>it<span class="token punctuation">.</span><span class="token function">hasNext</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span> <span class="token punctuation">{</span>
                <span class="token class-name">Writable</span> wt<span class="token operator">=</span>it<span class="token punctuation">.</span><span class="token function">next</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
                outkey<span class="token punctuation">.</span><span class="token function">add</span><span class="token punctuation">(</span>wt<span class="token punctuation">.</span><span class="token function">toString</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
                <span class="token class-name">Writable</span> wvalue<span class="token operator">=</span>value<span class="token punctuation">.</span><span class="token function">get</span><span class="token punctuation">(</span>wt<span class="token punctuation">)</span><span class="token punctuation">;</span>
                outvalue<span class="token punctuation">.</span><span class="token function">add</span><span class="token punctuation">(</span>wvalue<span class="token punctuation">.</span><span class="token function">toString</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
            <span class="token punctuation">}</span>


            <span class="token keyword">try</span> <span class="token punctuation">{</span>
                <span class="token keyword">if</span> <span class="token punctuation">(</span>outvalue<span class="token punctuation">.</span><span class="token function">get</span><span class="token punctuation">(</span>outkey<span class="token punctuation">.</span><span class="token function">indexOf</span><span class="token punctuation">(</span><span class="token string">&quot;type&quot;</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">equals</span><span class="token punctuation">(</span><span class="token string">&quot;q&quot;</span><span class="token punctuation">)</span><span class="token punctuation">)</span> <span class="token punctuation">{</span>
                    <span class="token class-name">String</span> down <span class="token operator">=</span> outvalue<span class="token punctuation">.</span><span class="token function">get</span><span class="token punctuation">(</span>outkey<span class="token punctuation">.</span><span class="token function">indexOf</span><span class="token punctuation">(</span><span class="token string">&quot;download_date&quot;</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
                    <span class="token class-name">String</span> s <span class="token operator">=</span> outvalue<span class="token punctuation">.</span><span class="token function">get</span><span class="token punctuation">(</span>outkey<span class="token punctuation">.</span><span class="token function">indexOf</span><span class="token punctuation">(</span><span class="token string">&quot;format_content&quot;</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span>

                    <span class="token comment">//被转发用户相关</span>
                    <span class="token class-name">String</span> forward_user_url <span class="token operator">=</span> outvalue<span class="token punctuation">.</span><span class="token function">get</span><span class="token punctuation">(</span>outkey<span class="token punctuation">.</span><span class="token function">indexOf</span><span class="token punctuation">(</span><span class="token string">&quot;forward_user_url&quot;</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
                    <span class="token class-name">String</span> forward_author <span class="token operator">=</span> outvalue<span class="token punctuation">.</span><span class="token function">get</span><span class="token punctuation">(</span>outkey<span class="token punctuation">.</span><span class="token function">indexOf</span><span class="token punctuation">(</span><span class="token string">&quot;forward_author&quot;</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
                    <span class="token class-name">String</span> forward_uid <span class="token operator">=</span> outvalue<span class="token punctuation">.</span><span class="token function">get</span><span class="token punctuation">(</span>outkey<span class="token punctuation">.</span><span class="token function">indexOf</span><span class="token punctuation">(</span><span class="token string">&quot;forward_uid&quot;</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
                    <span class="token class-name">String</span> verified <span class="token operator">=</span> outvalue<span class="token punctuation">.</span><span class="token function">get</span><span class="token punctuation">(</span>outkey<span class="token punctuation">.</span><span class="token function">indexOf</span><span class="token punctuation">(</span><span class="token string">&quot;verified&quot;</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span>

                    <span class="token comment">//被转发微博相关</span>
                    <span class="token class-name">String</span> refer_url <span class="token operator">=</span> outvalue<span class="token punctuation">.</span><span class="token function">get</span><span class="token punctuation">(</span>outkey<span class="token punctuation">.</span><span class="token function">indexOf</span><span class="token punctuation">(</span><span class="token string">&quot;refer_url&quot;</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
                    <span class="token class-name">String</span> post_source <span class="token operator">=</span> outvalue<span class="token punctuation">.</span><span class="token function">get</span><span class="token punctuation">(</span>outkey<span class="token punctuation">.</span><span class="token function">indexOf</span><span class="token punctuation">(</span><span class="token string">&quot;post_source&quot;</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
                    <span class="token class-name">String</span> forward_url <span class="token operator">=</span> outvalue<span class="token punctuation">.</span><span class="token function">get</span><span class="token punctuation">(</span>outkey<span class="token punctuation">.</span><span class="token function">indexOf</span><span class="token punctuation">(</span><span class="token string">&quot;forward_url&quot;</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
                    <span class="token class-name">Date</span> postdate <span class="token operator">=</span> <span class="token function">getPostDate</span><span class="token punctuation">(</span>down<span class="token punctuation">,</span> s<span class="token punctuation">)</span><span class="token punctuation">;</span>
                    <span class="token class-name">String</span> <span class="token class-name">Post</span> <span class="token operator">=</span> <span class="token function">getPostContent</span><span class="token punctuation">(</span>s<span class="token punctuation">)</span><span class="token punctuation">;</span>

                    <span class="token comment">//转发微博信息</span>
                    <span class="token class-name">Date</span> release_date <span class="token operator">=</span> <span class="token function">getRepostDate</span><span class="token punctuation">(</span>outvalue<span class="token punctuation">.</span><span class="token function">get</span><span class="token punctuation">(</span>outkey<span class="token punctuation">.</span><span class="token function">indexOf</span><span class="token punctuation">(</span><span class="token string">&quot;release_date&quot;</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
                    <span class="token class-name">String</span> title <span class="token operator">=</span> outvalue<span class="token punctuation">.</span><span class="token function">get</span><span class="token punctuation">(</span>outkey<span class="token punctuation">.</span><span class="token function">indexOf</span><span class="token punctuation">(</span><span class="token string">&quot;title&quot;</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
                    <span class="token class-name">String</span> content <span class="token operator">=</span> outvalue<span class="token punctuation">.</span><span class="token function">get</span><span class="token punctuation">(</span>outkey<span class="token punctuation">.</span><span class="token function">indexOf</span><span class="token punctuation">(</span><span class="token string">&quot;content&quot;</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
                    <span class="token class-name">String</span> comments_count <span class="token operator">=</span> outvalue<span class="token punctuation">.</span><span class="token function">get</span><span class="token punctuation">(</span>outkey<span class="token punctuation">.</span><span class="token function">indexOf</span><span class="token punctuation">(</span><span class="token string">&quot;comments_count&quot;</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
                    <span class="token class-name">String</span> url <span class="token operator">=</span> outvalue<span class="token punctuation">.</span><span class="token function">get</span><span class="token punctuation">(</span>outkey<span class="token punctuation">.</span><span class="token function">indexOf</span><span class="token punctuation">(</span><span class="token string">&quot;url&quot;</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
                    <span class="token class-name">String</span> attitudes_count <span class="token operator">=</span> outvalue<span class="token punctuation">.</span><span class="token function">get</span><span class="token punctuation">(</span>outkey<span class="token punctuation">.</span><span class="token function">indexOf</span><span class="token punctuation">(</span><span class="token string">&quot;attitudes_count&quot;</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
                    <span class="token class-name">String</span> quote_count <span class="token operator">=</span> outvalue<span class="token punctuation">.</span><span class="token function">get</span><span class="token punctuation">(</span>outkey<span class="token punctuation">.</span><span class="token function">indexOf</span><span class="token punctuation">(</span><span class="token string">&quot;quote_count&quot;</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span>

                    <span class="token comment">//发表者相关信息</span>
                    <span class="token class-name">String</span> author <span class="token operator">=</span> outvalue<span class="token punctuation">.</span><span class="token function">get</span><span class="token punctuation">(</span>outkey<span class="token punctuation">.</span><span class="token function">indexOf</span><span class="token punctuation">(</span><span class="token string">&quot;author&quot;</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
                    <span class="token class-name">String</span> user_url <span class="token operator">=</span> outvalue<span class="token punctuation">.</span><span class="token function">get</span><span class="token punctuation">(</span>outkey<span class="token punctuation">.</span><span class="token function">indexOf</span><span class="token punctuation">(</span><span class="token string">&quot;user_url&quot;</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
                    <span class="token class-name">String</span> uid <span class="token operator">=</span> outvalue<span class="token punctuation">.</span><span class="token function">get</span><span class="token punctuation">(</span>outkey<span class="token punctuation">.</span><span class="token function">indexOf</span><span class="token punctuation">(</span><span class="token string">&quot;uid&quot;</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span>

                    <span class="token comment">//媒体来源</span>
                    <span class="token class-name">String</span> media_id <span class="token operator">=</span> outvalue<span class="token punctuation">.</span><span class="token function">get</span><span class="token punctuation">(</span>outkey<span class="token punctuation">.</span><span class="token function">indexOf</span><span class="token punctuation">(</span><span class="token string">&quot;media_id&quot;</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
                    <span class="token class-name">String</span> media_name <span class="token operator">=</span> outvalue<span class="token punctuation">.</span><span class="token function">get</span><span class="token punctuation">(</span>outkey<span class="token punctuation">.</span><span class="token function">indexOf</span><span class="token punctuation">(</span><span class="token string">&quot;media_name&quot;</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span>

                    <span class="token comment">//入库</span>
                    <span class="token keyword">if</span> <span class="token punctuation">(</span>postdate <span class="token operator">!=</span> <span class="token keyword">null</span> <span class="token operator">&amp;&amp;</span> release_date <span class="token operator">!=</span> <span class="token keyword">null</span><span class="token punctuation">)</span> <span class="token punctuation">{</span>
                        <span class="token class-name">DateFormat</span> outdf <span class="token operator">=</span> <span class="token keyword">new</span> <span class="token class-name">SimpleDateFormat</span><span class="token punctuation">(</span><span class="token string">&quot;yyyy-MM-dd HH:mm&quot;</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
                        <span class="token keyword">byte</span><span class="token punctuation">[</span><span class="token punctuation">]</span> bRowKey <span class="token operator">=</span> <span class="token class-name">Bytes</span><span class="token punctuation">.</span><span class="token function">toBytes</span><span class="token punctuation">(</span>url<span class="token punctuation">)</span><span class="token punctuation">;</span>
                        <span class="token class-name">ImmutableBytesWritable</span> rowkey <span class="token operator">=</span> <span class="token keyword">new</span> <span class="token class-name">ImmutableBytesWritable</span><span class="token punctuation">(</span>bRowKey<span class="token punctuation">)</span><span class="token punctuation">;</span>
                        <span class="token class-name">Put</span> put <span class="token operator">=</span> <span class="token keyword">new</span> <span class="token class-name">Put</span><span class="token punctuation">(</span>bRowKey<span class="token punctuation">)</span><span class="token punctuation">;</span>


                        <span class="token comment">//&quot;forward_user&quot;列族</span>
                        put<span class="token punctuation">.</span><span class="token function">addImmutable</span><span class="token punctuation">(</span><span class="token string">&quot;forward_user&quot;</span><span class="token punctuation">.</span><span class="token function">getBytes</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token string">&quot;forward_user_url&quot;</span><span class="token punctuation">.</span><span class="token function">getBytes</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> forward_user_url<span class="token punctuation">.</span><span class="token function">getBytes</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
                        put<span class="token punctuation">.</span><span class="token function">addImmutable</span><span class="token punctuation">(</span><span class="token string">&quot;forward_user&quot;</span><span class="token punctuation">.</span><span class="token function">getBytes</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token string">&quot;forward_author&quot;</span><span class="token punctuation">.</span><span class="token function">getBytes</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> forward_author<span class="token punctuation">.</span><span class="token function">getBytes</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
                        put<span class="token punctuation">.</span><span class="token function">addImmutable</span><span class="token punctuation">(</span><span class="token string">&quot;forward_user&quot;</span><span class="token punctuation">.</span><span class="token function">getBytes</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token string">&quot;forward_uid&quot;</span><span class="token punctuation">.</span><span class="token function">getBytes</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> forward_uid<span class="token punctuation">.</span><span class="token function">getBytes</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
                        put<span class="token punctuation">.</span><span class="token function">addImmutable</span><span class="token punctuation">(</span><span class="token string">&quot;forward_user&quot;</span><span class="token punctuation">.</span><span class="token function">getBytes</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token string">&quot;verified&quot;</span><span class="token punctuation">.</span><span class="token function">getBytes</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> verified<span class="token punctuation">.</span><span class="token function">getBytes</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span>

                        <span class="token comment">//&quot;forward_weibo&quot;列族</span>
                        put<span class="token punctuation">.</span><span class="token function">addImmutable</span><span class="token punctuation">(</span><span class="token string">&quot;forward_weibo&quot;</span><span class="token punctuation">.</span><span class="token function">getBytes</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token string">&quot;refer_url&quot;</span><span class="token punctuation">.</span><span class="token function">getBytes</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> refer_url<span class="token punctuation">.</span><span class="token function">getBytes</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
                        put<span class="token punctuation">.</span><span class="token function">addImmutable</span><span class="token punctuation">(</span><span class="token string">&quot;forward_weibo&quot;</span><span class="token punctuation">.</span><span class="token function">getBytes</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token string">&quot;post_source&quot;</span><span class="token punctuation">.</span><span class="token function">getBytes</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> post_source<span class="token punctuation">.</span><span class="token function">getBytes</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
                        put<span class="token punctuation">.</span><span class="token function">addImmutable</span><span class="token punctuation">(</span><span class="token string">&quot;forward_weibo&quot;</span><span class="token punctuation">.</span><span class="token function">getBytes</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token string">&quot;forward_url&quot;</span><span class="token punctuation">.</span><span class="token function">getBytes</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> forward_url<span class="token punctuation">.</span><span class="token function">getBytes</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
                        put<span class="token punctuation">.</span><span class="token function">addImmutable</span><span class="token punctuation">(</span><span class="token string">&quot;forward_weibo&quot;</span><span class="token punctuation">.</span><span class="token function">getBytes</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token string">&quot;postdate&quot;</span><span class="token punctuation">.</span><span class="token function">getBytes</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> outdf<span class="token punctuation">.</span><span class="token function">format</span><span class="token punctuation">(</span>postdate<span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">getBytes</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
                        put<span class="token punctuation">.</span><span class="token function">addImmutable</span><span class="token punctuation">(</span><span class="token string">&quot;forward_weibo&quot;</span><span class="token punctuation">.</span><span class="token function">getBytes</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token string">&quot;Post&quot;</span><span class="token punctuation">.</span><span class="token function">getBytes</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token class-name">Post</span><span class="token punctuation">.</span><span class="token function">getBytes</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span>


                        <span class="token comment">//&quot;post&quot;列族</span>
                        put<span class="token punctuation">.</span><span class="token function">addImmutable</span><span class="token punctuation">(</span><span class="token string">&quot;post&quot;</span><span class="token punctuation">.</span><span class="token function">getBytes</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token string">&quot;release_date&quot;</span><span class="token punctuation">.</span><span class="token function">getBytes</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> outdf<span class="token punctuation">.</span><span class="token function">format</span><span class="token punctuation">(</span>release_date<span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">getBytes</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
                        put<span class="token punctuation">.</span><span class="token function">addImmutable</span><span class="token punctuation">(</span><span class="token string">&quot;post&quot;</span><span class="token punctuation">.</span><span class="token function">getBytes</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token string">&quot;title&quot;</span><span class="token punctuation">.</span><span class="token function">getBytes</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> title<span class="token punctuation">.</span><span class="token function">getBytes</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
                        put<span class="token punctuation">.</span><span class="token function">addImmutable</span><span class="token punctuation">(</span><span class="token string">&quot;post&quot;</span><span class="token punctuation">.</span><span class="token function">getBytes</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token string">&quot;content&quot;</span><span class="token punctuation">.</span><span class="token function">getBytes</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> content<span class="token punctuation">.</span><span class="token function">getBytes</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
                        put<span class="token punctuation">.</span><span class="token function">addImmutable</span><span class="token punctuation">(</span><span class="token string">&quot;post&quot;</span><span class="token punctuation">.</span><span class="token function">getBytes</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token string">&quot;comments_count&quot;</span><span class="token punctuation">.</span><span class="token function">getBytes</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> comments_count<span class="token punctuation">.</span><span class="token function">getBytes</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
                        put<span class="token punctuation">.</span><span class="token function">addImmutable</span><span class="token punctuation">(</span><span class="token string">&quot;post&quot;</span><span class="token punctuation">.</span><span class="token function">getBytes</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token string">&quot;url&quot;</span><span class="token punctuation">.</span><span class="token function">getBytes</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> url<span class="token punctuation">.</span><span class="token function">getBytes</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
                        put<span class="token punctuation">.</span><span class="token function">addImmutable</span><span class="token punctuation">(</span><span class="token string">&quot;post&quot;</span><span class="token punctuation">.</span><span class="token function">getBytes</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token string">&quot;attitudes_count&quot;</span><span class="token punctuation">.</span><span class="token function">getBytes</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> attitudes_count<span class="token punctuation">.</span><span class="token function">getBytes</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
                        put<span class="token punctuation">.</span><span class="token function">addImmutable</span><span class="token punctuation">(</span><span class="token string">&quot;post&quot;</span><span class="token punctuation">.</span><span class="token function">getBytes</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token string">&quot;quote_count&quot;</span><span class="token punctuation">.</span><span class="token function">getBytes</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> quote_count<span class="token punctuation">.</span><span class="token function">getBytes</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span>


                        <span class="token comment">//&quot;poster&quot;列族</span>
                        put<span class="token punctuation">.</span><span class="token function">addImmutable</span><span class="token punctuation">(</span><span class="token string">&quot;poster&quot;</span><span class="token punctuation">.</span><span class="token function">getBytes</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token string">&quot;author&quot;</span><span class="token punctuation">.</span><span class="token function">getBytes</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> author<span class="token punctuation">.</span><span class="token function">getBytes</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
                        put<span class="token punctuation">.</span><span class="token function">addImmutable</span><span class="token punctuation">(</span><span class="token string">&quot;poster&quot;</span><span class="token punctuation">.</span><span class="token function">getBytes</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token string">&quot;user_url&quot;</span><span class="token punctuation">.</span><span class="token function">getBytes</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> user_url<span class="token punctuation">.</span><span class="token function">getBytes</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
                        put<span class="token punctuation">.</span><span class="token function">addImmutable</span><span class="token punctuation">(</span><span class="token string">&quot;poster&quot;</span><span class="token punctuation">.</span><span class="token function">getBytes</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token string">&quot;uid&quot;</span><span class="token punctuation">.</span><span class="token function">getBytes</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> uid<span class="token punctuation">.</span><span class="token function">getBytes</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span>


                        <span class="token comment">//&quot;media&quot;列族</span>
                        put<span class="token punctuation">.</span><span class="token function">addImmutable</span><span class="token punctuation">(</span><span class="token string">&quot;media&quot;</span><span class="token punctuation">.</span><span class="token function">getBytes</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token string">&quot;media_id&quot;</span><span class="token punctuation">.</span><span class="token function">getBytes</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> media_id<span class="token punctuation">.</span><span class="token function">getBytes</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
                        put<span class="token punctuation">.</span><span class="token function">addImmutable</span><span class="token punctuation">(</span><span class="token string">&quot;media&quot;</span><span class="token punctuation">.</span><span class="token function">getBytes</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> <span class="token string">&quot;media_name&quot;</span><span class="token punctuation">.</span><span class="token function">getBytes</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">,</span> media_name<span class="token punctuation">.</span><span class="token function">getBytes</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">;</span>

                        context<span class="token punctuation">.</span><span class="token function">write</span><span class="token punctuation">(</span>rowkey<span class="token punctuation">,</span> put<span class="token punctuation">)</span><span class="token punctuation">;</span>
                    <span class="token punctuation">}</span>
                <span class="token punctuation">}</span>
            <span class="token punctuation">}</span><span class="token keyword">catch</span> <span class="token punctuation">(</span><span class="token class-name">Exception</span> ex<span class="token punctuation">)</span>
            <span class="token punctuation">{</span>

            <span class="token punctuation">}</span>

        <span class="token punctuation">}</span>
    <span class="token punctuation">}</span>
    <span class="token keyword">public</span> <span class="token keyword">static</span> <span class="token keyword">void</span> <span class="token function">main</span><span class="token punctuation">(</span><span class="token class-name">String</span><span class="token punctuation">[</span><span class="token punctuation">]</span> args<span class="token punctuation">)</span> <span class="token keyword">throws</span> <span class="token class-name">Exception</span> <span class="token punctuation">{</span>
        <span class="token class-name">System</span><span class="token punctuation">.</span><span class="token function">setProperty</span><span class="token punctuation">(</span><span class="token string">&quot;hadoop.home.dir&quot;</span><span class="token punctuation">,</span><span class="token string">&quot;C:\\hadoop&quot;</span> <span class="token punctuation">)</span><span class="token punctuation">;</span>
        <span class="token class-name">Configuration</span> conf <span class="token operator">=</span> <span class="token class-name">HBaseConfiguration</span><span class="token punctuation">.</span><span class="token function">create</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
        conf<span class="token punctuation">.</span><span class="token function">set</span><span class="token punctuation">(</span><span class="token string">&quot;hbase.zookeeper.quorum&quot;</span><span class="token punctuation">,</span> <span class="token string">&quot;TjuBD&quot;</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
        conf<span class="token punctuation">.</span><span class="token function">set</span><span class="token punctuation">(</span><span class="token string">&quot;hbase.zookeeper.property.clientPort&quot;</span><span class="token punctuation">,</span> <span class="token string">&quot;2181&quot;</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
        conf<span class="token punctuation">.</span><span class="token function">set</span><span class="token punctuation">(</span><span class="token string">&quot;dfs.socket.timeout&quot;</span><span class="token punctuation">,</span> <span class="token string">&quot;180000&quot;</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
        <span class="token class-name">Path</span> seqFile <span class="token operator">=</span> <span class="token keyword">new</span> <span class="token class-name">Path</span><span class="token punctuation">(</span><span class="token string">&quot;hdfs://172.28.9.62:8020/weibo/201701&quot;</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
        conf<span class="token punctuation">.</span><span class="token function">set</span><span class="token punctuation">(</span><span class="token string">&quot;io.compression.codecs&quot;</span><span class="token punctuation">,</span> <span class="token string">&quot;com.hadoop.compression.lzo.LzoCodec&quot;</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
        conf<span class="token punctuation">.</span><span class="token function">set</span><span class="token punctuation">(</span><span class="token string">&quot;fs.default.name&quot;</span><span class="token punctuation">,</span> <span class="token string">&quot;hdfs://172.28.9.62:8020&quot;</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
        conf<span class="token punctuation">.</span><span class="token function">set</span><span class="token punctuation">(</span><span class="token string">&quot;mapreduce.input.fileinputformat.input.dir.recursive&quot;</span><span class="token punctuation">,</span> <span class="token string">&quot;True&quot;</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
        conf<span class="token punctuation">.</span><span class="token function">set</span><span class="token punctuation">(</span><span class="token string">&quot;hadoop.job.user&quot;</span><span class="token punctuation">,</span> <span class="token string">&quot;hadoop&quot;</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
        conf<span class="token punctuation">.</span><span class="token function">set</span><span class="token punctuation">(</span><span class="token string">&quot;mapreduce.framework.name&quot;</span><span class="token punctuation">,</span> <span class="token string">&quot;yarn&quot;</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
        conf<span class="token punctuation">.</span><span class="token function">set</span><span class="token punctuation">(</span><span class="token string">&quot;mapreduce.jobtracker.address&quot;</span><span class="token punctuation">,</span> <span class="token string">&quot;172.28.9.62:9001&quot;</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
        conf<span class="token punctuation">.</span><span class="token function">set</span><span class="token punctuation">(</span><span class="token string">&quot;yarn.resourcemanager.hostname&quot;</span><span class="token punctuation">,</span> <span class="token string">&quot;172.28.9.62&quot;</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
        conf<span class="token punctuation">.</span><span class="token function">set</span><span class="token punctuation">(</span><span class="token string">&quot;yarn.resourcemanager.admin.address&quot;</span><span class="token punctuation">,</span> <span class="token string">&quot;172.28.9.62:8033&quot;</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
        conf<span class="token punctuation">.</span><span class="token function">set</span><span class="token punctuation">(</span><span class="token string">&quot;yarn.resourcemanager.address&quot;</span><span class="token punctuation">,</span> <span class="token string">&quot;172.28.9.62:8032&quot;</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
        conf<span class="token punctuation">.</span><span class="token function">set</span><span class="token punctuation">(</span><span class="token string">&quot;mapreduce.job.jar&quot;</span><span class="token punctuation">,</span> <span class="token string">&quot;C:\\Users\\lcc\\Desktop\\readfile\\target\\readfile-1.0-SNAPSHOT-jar-with-dependencies.jar&quot;</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
        conf<span class="token punctuation">.</span><span class="token function">set</span><span class="token punctuation">(</span><span class="token string">&quot;yarn.resourcemanager.resource-tracker.address&quot;</span><span class="token punctuation">,</span> <span class="token string">&quot;172.28.9.62:8036&quot;</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
        conf<span class="token punctuation">.</span><span class="token function">set</span><span class="token punctuation">(</span><span class="token string">&quot;mapreduce.app-submission.cross-platform&quot;</span><span class="token punctuation">,</span> <span class="token string">&quot;true&quot;</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
        conf<span class="token punctuation">.</span><span class="token function">set</span><span class="token punctuation">(</span><span class="token string">&quot;yarn.resourcemanager.scheduler.address&quot;</span><span class="token punctuation">,</span> <span class="token string">&quot;172.28.9.62:8030&quot;</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
        <span class="token class-name">Job</span> job <span class="token operator">=</span> <span class="token class-name">Job</span><span class="token punctuation">.</span><span class="token function">getInstance</span><span class="token punctuation">(</span>conf<span class="token punctuation">,</span> <span class="token string">&quot;FromWeiboToHbase&quot;</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
        job<span class="token punctuation">.</span><span class="token function">setJarByClass</span><span class="token punctuation">(</span><span class="token class-name">FromWeibo</span><span class="token punctuation">.</span><span class="token keyword">class</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
        job<span class="token punctuation">.</span><span class="token function">setInputFormatClass</span><span class="token punctuation">(</span><span class="token class-name">SequenceFileInputFormat</span><span class="token punctuation">.</span><span class="token keyword">class</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
        <span class="token class-name">FileInputFormat</span><span class="token punctuation">.</span><span class="token function">setInputPaths</span><span class="token punctuation">(</span>job<span class="token punctuation">,</span> seqFile<span class="token punctuation">)</span><span class="token punctuation">;</span>
        job<span class="token punctuation">.</span><span class="token function">setMapperClass</span><span class="token punctuation">(</span><span class="token class-name">ToHbaseMap</span><span class="token punctuation">.</span><span class="token keyword">class</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
        <span class="token class-name">TableMapReduceUtil</span><span class="token punctuation">.</span><span class="token function">initTableReducerJob</span><span class="token punctuation">(</span><span class="token string">&quot;weibo&quot;</span><span class="token punctuation">,</span><span class="token keyword">null</span><span class="token punctuation">,</span>job<span class="token punctuation">)</span><span class="token punctuation">;</span>
        job<span class="token punctuation">.</span><span class="token function">setNumReduceTasks</span><span class="token punctuation">(</span><span class="token number">0</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
        <span class="token class-name">TableMapReduceUtil</span><span class="token punctuation">.</span><span class="token function">addDependencyJars</span><span class="token punctuation">(</span>job<span class="token punctuation">)</span><span class="token punctuation">;</span>
        <span class="token class-name">System</span><span class="token punctuation">.</span><span class="token function">exit</span><span class="token punctuation">(</span>job<span class="token punctuation">.</span><span class="token function">waitForCompletion</span><span class="token punctuation">(</span><span class="token boolean">true</span><span class="token punctuation">)</span><span class="token operator">?</span><span class="token number">0</span><span class="token operator">:</span><span class="token number">1</span><span class="token punctuation">)</span><span class="token punctuation">;</span>

    <span class="token punctuation">}</span>
<span class="token punctuation">}</span>

</code></pre></div></div> <div class="content-time" data-v-a5c9dc12><time datetime="2017年12月28日" class="time-text" data-v-a5c9dc12>Create Time: 2017年12月28日
    </time> <time datetime="2020年9月13日" class="time-text" data-v-a5c9dc12>Last Updated: 2020年9月13日
    </time></div></article> <section class="flex-xb main info-nav" data-v-e08c9474 data-v-52fe94f0><a href="/2017/12/27/%E4%BB%8Emongodb%E6%8A%BD%E5%8F%96%E6%95%B0%E6%8D%AE%E5%AF%BC%E5%85%A5mysql/" class="flex-xb nav-item" data-v-e08c9474><div class="flex-xcc item-img" data-v-e08c9474><img data-src="/images/8.jpg" loading="lazy" alt="从MongoDB抽取数据导入mysql" class="img lazy" data-v-e08c9474></div> <article class="flex-ysc item-content" data-v-e08c9474><h2 class="content-title" data-v-e08c9474>从MongoDB抽取数据导入mysql</h2> <div class="content" data-v-e08c9474></div></article></a> <a href="/2017/12/27/%E4%BB%8Ehdfs%E4%B8%8A%E8%AF%BB%E5%8F%96%E5%B8%A6lzo%E5%8E%8B%E7%BC%A9%E7%9A%84sequencefile%E6%96%87%E4%BB%B6/" class="flex-xb nav-item" data-v-e08c9474><div class="flex-xcc item-img" data-v-e08c9474><img data-src="/images/7.jpg" loading="lazy" alt="从HDFS上读取带lzo压缩的SequenceFile文件" class="img lazy" data-v-e08c9474></div> <article class="flex-ysc item-content" data-v-e08c9474><h2 class="content-title" data-v-e08c9474>从HDFS上读取带lzo压缩的SequenceFile文件</h2> <div class="content" data-v-e08c9474></div></article></a></section> <!----></section></section> <footer class="footer" data-v-5bc4f524 data-v-4fb7124e><nav class="link-list" data-v-5bc4f524><a href="https://gitee.com/tjuchaochao" target="_blank" rel="noopener noreferrer" class="list-item" data-v-5bc4f524>gitee</a></nav> <a href="/" class="copyright router-link-active" data-v-5bc4f524>智能技术 © 2021</a></footer></section><div class="global-ui"><!----><!----></div></div>
    <script src="/assets/js/app.cee3c598.js" defer></script><script src="/assets/js/13.16b64097.js" defer></script>
  </body>
</html>
